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Abstract 


Computer  vision  deals  with  algorithms  that  allow  machines  to  detect,  segment,  feature 
extract,  and  recognize  objects  in  an  image.  There  are  numerous  applications  in  medicine, 
manufacturing,  and  security  for  this  technology.  By  studying  the  visual  processes  of  biological 
systems,  enhancements  can  be  achieved  in  the  development  of  computer  vision  algorithms.  One 
biological  function  of  interest  involves  the  oscillatory  pulses  generated  in  the  primary  visual 
cortex  engaged  in  stimulus-specific  oscillatory  responses.  As  a  result  of  these  experiments,  it 
can  be  concluded  that  these  tightly  correlated,  stimulus-induced  oscillations  may  play  a  role  in 
the  recognition  of  images.  Therefore,  these  cortical  oscillations  have  been  modeled  to 
investigate  their  ability  to  segment  objects  in  a  visual  field.  This  report  briefly  discusses  the 
visual  system  and  the  internally  stimulus-dependent  oscillations  that  may  lead  to  identification 
of  images.  Emphasis  will  be  on  the  models  that  attempt  to  reproduce  this  biological  phtmomcin^^, 
their  computational  and  behavioral  aspects,  as  well  as  simulation  performance.  Detail  will  be 
given  to  their  computational  and  behavioral  aspects  since  it  is  in  these  areas  that  possible 
improvements  can  be  achieved  through  more  detailed  modeling. 
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1.  Introduction 


Computer  vision  deals  with  algorithms  that  allow  machines  to  detect,  segment,  feature  extract, 
and  recogmze  objects  in  an  image.  The  algorithms  can  then  be  implemented  and  transferred  to 
integrated  circuits  and  allow  machines  to  “see.”  There  are  numerous  applications  in  medicine, 
manufacturing,  and  security  for  this  technology.  Studying  the  visual  processes  of  biological  systems 
can  greatly  enhance  the  development  of  computer  vision  algorithms,  by  incorporating  biological 
functions  that  perform  tasks  so  efficiently.  Recently,  oscillations  in  the  brains  of  cats  due  to  visual 
stimuli  have  led  to  the  development  of  several  models  that  mimic  the  interacting  neurons.  Results 
of  this  work  may  lead  to  a  computer  model  that  segments  objects  in  an  image.  This  paper  presents 
how  biological  exploration  may  be  leading  to  tools  that  can  improve  advancements  in  the  area  of 
computer  vision. 

The  primary  visual  cortex  produces  oscillatory  pulses  in  response  to  specific  stimulus.  These 
responses  are  thought  to  be  produced  at  the  primary  visual  cortex  and  not  induced  by  other  sections 
of  the  visual  pathway  (see  Figure  1)  (Kandell,  Schwartz,  and  Jessell  1991;  Zeki  1990).  Therefore, 
these  cortical  oscillations  have  been  modeled  to  investigate  their  ability  to  recognize  objects  in  a 
visual  field.  There  are  three  models  that  can  be  used  to  reproduce  these  oscillations,  and  they  will 
be  discussed  in  the  following  sections. 

In  areas  17  (A17)  and  18  (A18)  of  the  cat  visual  cortex  (see  Figure  2)  (Zeki  1993),  the  firing  of 
neurons  in  response  to  the  presentation  of  optimally  aligned  light  bars  within  their  receptive  field 
oscillates  with  a  peak  frequency  near  40  Hz.  Thus,  neuronal  firing  pattern  is  tightly  correlated  with 
the  phase  and  amplitude  of  an  osciUatory  local  field  potential  (LFP)  recorded  from  the  same 
electrode.  Experiments  have  demonstrated  that  local  neuronal  populations  in  the  visual  cortex 
engaged  in  stimulus  specific  synchronous  oscillations  resulting  from  an  intracortical  mechanism. 

Moreover,  observations  in  A17  of  alert  cats  seem  to  indicate  that  the  neuronal  response  recorded 
during  periods  of  attention  exhibits  a  rhythmic  firing  pattern  that  is  tightly  correlated  with  an 
oscillatory  IFP  having  a  frequency  near  40  Hz.  Furthermore,  observations  and  reports  show  that 
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Figure  2.  The  Primary  Visual  Cortex  and  Brodmann’s  Cytoarchitectonic  Chart  (a)  and  the 
On-Center,  Off-Surround  and  the  Linear  Receptive  Fields  (b)  (continued). 


local  groups  of  neurons  within  functional  columns  of  the  visual  cortex  engaged  in  stimulus-specific 
oscillatory  responses  having  a  frequency  near  40  Hz.  These  results  suggest  fliat  the  temporal  patterns 
of  the  oscillatory  response  can  be  used  to  synchronize  the  activity  of  neuronal  populations  in 
spatially  separate  regions  of  die  cortex. 


As  a  result  of  the  experimentation  described,  it  can  be  concluded  that  these  tightly  correlated 
stimulus-  induced  oscillations  may  play  a  role  in  recognition  of  images  (stimuli).  Specifically,  if 
these  40-Hz  oscillations  are  specific  in  reference  to  the  applied  stimulus,  then  a  “map”  is  formed 
between  the  stimulus  and  the  subsequent  neurons  that  fire.  Therefore,  the  observation  of  the  firing 
pattern  of  the  neurons  may  be  used  to  “identify”  the  stimulus  that  was  applied  to  the  visual  field. 

A  brief  look  at  the  visual  system  will  be  presented  in  this  report.  Emphasis  is  on  those  areas 
leading  to  where  these  internally  stimulus-dependent  oscillations  occur,  the  biological  testing  done 
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to  observe  the  oscillations,  and  the  models  that  attempt  to  reproduce  this  biological  phenomena. 
This  information  is  used  as  the  basis  for  a  model  that  attempts  to  utilize  these  neuronal  responses  for 
segmentation  of  objects  in  an  image. 

This  report  is  oiganized  as  follows:  (1)  a  general  overview  of  the  anatomy  of  the  visual  system; 
(2)  a  discussion  of  the  biological  test  conducted  to  observe  the  cortical  oscillations;  (3)  an 
explanation  of  the  models  developed  to  simulate  tfie  cortical  oscillations  found  in  the  visual  system; 
and  (4)  a  discussion  of  questions  and  areas  of  further  research. 

2.  General  Anatomy  of  the  Visual  System 

Li  order  to  determine  how  the  visual  system  represents  images  in  the  visual  field  and  what 
responses  are  initiated  in  the  primary  visual  cortex,  a  general  overview  of  the  visual  system  is 
presented  in  succeeding  text.  The  parts  of  the  visual  system  discussed  are  the  visual  field,  the  visual 
pathway,  and  the  primary  visual  cortex. 

The  visual  field  is  the  view  seen  by  two  eyes  without  movement  of  die  head.  The  left  half  of  the 
visual  field  projects  to  the  nasal  hemiretina  of  the  left  eye  and  on  the  temporal  hemiretina  of  the  right 
eye,  and  similarly  for  the  right  half  of  the  visual  field  (see  Figure  1)  (Kandell,  Schwartz,  and  Jessell 
1991).  In  addition,  the  lens  of  the  eye  inverts  the  visual  image  that  is  projected  onto  the  back  of  the 
retina. 

The  optic  disc  is  located  in  the  back  of  the  retina  below  the  fovea.  It  can  be  pinpointed  as  the 
region  where  the  ganglion  cells  leave  the  retina.  These  ganglion  cells  form  the  optic  nerve.  The 
optic  nerve  from  one  eye  joins  at  the  optic  chiasm  with  that  of  the  other,  which  allows  the  nerves  to 
continue  traversing  the  brain.  Fibers  from  both  eyes  enter  the  optic  tract  and  project  to  the  lateral 
geniculate  nucleus  (LGN).  The  LGN  is  a  section  of  die  cerebral  hemisphere.  Specifically,  the  fibers 
from  the  nasal  half  of  each  retina  cross  to  opposite  sides  of  the  brain  and  axons  from  the  temporal 
half  project  to  the  same  side  of  the  brain  to  the  LGN.  Tlierefore,  the  left  optic  tract  is  the  section  of 
the  pathway  that  carries  a  complete  representation  of  the  right  half  of  the  visual  field  to  the  LGN  and 
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similarly  for  the  left.  The  LGN  has  six  layers  of  cell  bodies.  The  ventral  layers  contain  large  cell 
bodies  and,  therefore,  is  called  magnocellular  (M)  layers.  The  dorsal  layers  contain  the  sman  cell 
bodies  and  is  called  parvocelluar  (P)  layers.  The  M  cells  and  die  P  cells  project  to  the  primary  visual 
cortex.  The  primary  visual  cortex  is  hidden  within  the  calcarine  sulcus  area  on  the  medial  surface 
of  die  hemisphere  (see  Figure  2a).  The  primary  visual  cortex  represents  the  visual  field  by 
transforming  receptive  fields  into  linear  segments  and  boundaries.  The  visual  cortex  is  subdivided 
anatomically  into  five  areas,  labeled  V1-V5.  The  primary  visual  cortex  is  considered  VI  and  V2, 
also  known  as  Areas  17  and  18  (Brodmann’s  cytoarchitectonic  chart  of  the  brain). 

The  cells  found  in  the  primary  visual  cortex  are  pyramidal  and  stellate  cells.  Pyramidal  cells  are 
large  and  have  long  spiny  dendrites.  They  project  to  other  areas  of  the  brain.  On  the  other  hand,  the 
stellate  cells  have  spiny  and  smooth  dendrites.  They  are  found  solely  in  the  primary  visual  cortex. 
The  pyramidal  and  spiny  stellate  cells  are  excitatory  and  use  glutamate  or  aspartate  as  their 
neurotransmitter.  The  smooth  stellate  cells  are  inhibitory  cells  and  use  GABA  as  its 
neurotransmitter.  Once  nerves  from  LGN  enter  the  primary  visual  cortex,  information  flows  from 
one  cortical  layer  to  another,  starting  with  spiny  stellate  cells.  The  spiny  stellate  cells  project  to  the 
pyraimdal  cells  to  form  the  excitatory  path.  The  pyramidal  cells  excite  inhibitory,  smooth  stellate 
cells,  modulating  the  firing  of  the  excitatory  cells. 

Several  stimuli  could  be  used  to  mediate  the  firing  of  these  cells.  However,  experiments 
that  a  small  spot  of  light  is  the  most  effective  stimulus  for  the  retina,  the  LGN,  and  the  input  layer 
of  the  cortex.  Neurons  in  the  LGN  and  retina  are  known  to  have  an  on-center,  off-surround  receptive 
field  (see  Figure  2b).  For  an  on-center,  off-surround  receptive  field  application  of  diffused  light 
initiates  no  response;  if  light  is  shown  in  the  center,  a  response  is  initiated.  If  no  light  is  shown  in 
the  surround  area  (outside  the  smaller  circle  but  inside  the  larger  circle),  a  response  is  initiated. 
These  responses  are  the  firing  of  the  cells  in  the  visual  cortex. 

Cells  in  the  primary  visual  cortex  do  not  have  circular  receptive  fields,  and  they  respond  best  to 
stimulus  that  is  linear.  These  cells  are  divided  into  simple  and  complex  cells.  Simple  cells  and 
complex  cells  have  receptive  fields  that  have  a  specific  axis  of  orientation.  A  cell  can  have  a 
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rectangular  receptive  field  where  one  area  is  excitatory  and  the  oth^  area  is  inhibitory.  Complex 
cells  respond  most  effectively  to  a  moving  bar  of  light.  Thus,  a  stimulus  must  excite  a  segment  of 
the  retina  and  have  a  specific  axis  of  oiiratation.  A  bar  of  light  widi  a  particular  orientation  or  a  spot 
of  light  initiates  a  response  by  exciting  several  nemons.  When  the  orientation  of  the  bar  of  light  or 
the  location  of  the  q)ot  of  light  is  changed,  then  the  response  is  diminished  by  the  exciting  inhibitory 
neurons. 

The  ovCTview  of  the  anatomy  is  needed  as  background  to  understand  how  the  visual  system 
functions.  Now  that  the  general  structure  of  the  visual  system  is  presented,  our  attention  can  be 
focused  on  the  primary  visual  cortex  where  the  oscillations  occur.  The  biological  test  and 
subsequent  models  focus  on  the  response  observed  in  the  primary  visual  cortex.  The  models  are 
empirical  in  nature  and  do  not  model  the  visual  cortex  itself. 

3.  Biological  Testing 


The  biological  tests  were  experiments  performed  on  cats.  The  cats  were  prepared  for  surgery  by 
injecting  short-acting  anesthesia  that  was  maintained  during  recordings.  Recordings  were  taVm 
3-4  hrs  after  the  initial  anesthesia  to  ensure  any  negative  affects  had  sufficiently  worn  off.  The 
neuronal  activity  and  voltage  readings  were  recorded  from  25-pm-diameter  electrodes  (see 
Figure  3a)  (Gray  and  Singer  1989;  Leibovic  1990). 

This  section  examines  the  biological  tests  that  were  performed  in  A17  and  A18.  The  general  test 
was  conducted  by  placing  electrodes  in  the  primary  visual  cortex.  A  horizontal  bar  of  light  moving 
vertically  down  was  used  as  the  stimulus.  For  this  stimulus,  an  increase  in  voltage  was  measured 
and  an  excitatory  response  occurred.  When  the  bar  of  light  was  moved  vertically  up,  an  excitatory 
response  was  measured  also.  For  that  orientation  of  the  stimulus,  a  positive  voltage  and  an  excitatory 
response  occurred.  When  a  vadcal  bar  of  li^t  moving  horizontally  to  the  right  was  used,  a  negative 
voltage  was  measured  and  an  inhibitory  response  was  elicited.  When  the  bar  of  light  was  moved 
horizontally  to  the  left,  an  inhibitory  potential  was  measured  again.  For  this  particular  orientation, 
a  negative  voltage  and  an  inhibitory  potential  were  measured.  If  the  stimulus  is  removed,  the 
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Figure  3.  Electrodes  in  the  Primary  Visual  Cortex  and  the  Responses  to  Specific  Stimuli 
(a)  and  Multiunit  Activity  (MUA)  and  LFP  (b). 
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Figure  3.  Electrodes  in  the  Primary  Visual  Cortex  and  the  Responses  to  Specific  Stimuli 
(a)  and  Multiunit  Activity  (MUA)  and  LFP  (b)  (continued). 
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response,  whether  excitatory  or  inhibitory,  is  removed.  Hence,  a  spot  of  light  or  a  moving  bar  of 
light  can  be  used  as  a  stimulus  to  elicit  firing  of  neurons  in  the  primary  visual  cortex.  Therefore,  the 
presence  of  an  effective  stimulus  results  in  the  firing  of  the  corresponding  neurons. 

The  experiments  performed  by  Gray  and  Singer  also  involved  placing  electrodes  in  the  visual 
cortex  of  a  cat  A  stimulus  was  applied  in  the  visual  field,  and  recordings  of  neuronal  responses  in 
the  visual  cortex  were  taken.  The  receptive  field  properties  of  the  neurons  firing  were  recorded  fiom 
each  electrode  with  a  bar  of  light  projected  on  a  screen  in  firont  of  the  cat’s  eye  plane.  The 
measurements  taken  from  the  electrode  include  the  LFP,  which  is  die  average  voltage,  and  the  MUA, 
which  indicates  the  neurons  firing  (see  Figure  3b). 

The  recordings  from  areas  17  and  18  in  an  adult  cat  when  a  light  bar  of  optimal  orientation  was 
passed  through  the  receptive  field  show  a  rhythmic  firing  pattern.  This  neuronal  spike  train  was 
associated  with  a  high-amplitude  oscillation  of  the  LFP.  The  spike  occurred  during  the  negative 
phase  of  the  LFP  oscillations,  indicating  periodicity  in  the  firing  of  neurons.  It  is  noted  that  the 
highest  frequency  of  the  LFP  directly  correlates  with  the  maximum  number  of  neurons  firing  in  the 
MUA.  Computation  of  the  power  spectra  of  the  LFP  was  used  to  support  that  the  oscillatory 
activities  of  the  neurons  were  stimulus-  dependent. 

The  average  peak  frequency  recorded  of  the  LFP  was  between  35  and  49  Hz.  It  was  observed 
that  the  amplitude  and  latency  of  the  response  are  similar  for  the  LFP  and  the  MUA.  It  was  found 
that  oscillatory  responses  occurred  much  more  fiiequently  in  the  complex  cells  as  opposed  to  the 
simple  cells  of  the  primary  visual  cortex. 


The  results  of  these  experiments  suggest  that  during  responses  to  light  stimulus,  adjacent  neurons 
have  a  strong  tendency  to  act  simultaneously  and  synchronously.  These  oscillations  result  from 
intracortical  mechanisms  and  not  by  oscillatory  inputs  firom  the  LGN.  In  addition,  results  suggest 
that  the  synchronous  oscillation  of  ensemble  neurons  in  the  frequency  range  of  35-50  Hz  is  an 
integral  part  of  the  neuronal  response  in  the  visual  cortex. 
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In  addition,  the  field  potratial  has  revealed  the  oscillatory  behavior  of  the  neuronal  response,  and 
because  of  its  close  correlation  to  the  MUA,  this  field  potential  seems  to  reflect  the  synchronous 
activity  of  a  population  of  cells. 

Although  die  mechanism  of  the  oscillations  was  unknown,  it  suggested  that  the  interaction 
among  a  population  of  synaptically  coupled  neurons,  both  excitation  and  recurrent  inhibition,  was 
sufficient  for  generating  oscillations.  Experiments  also  show  that  when  activated  appropriately, 
groups  of  adjacent  cortical  neurons  change  in  cooperative  interactions.  These  interactions  led  to 
coherent  and  periodic  patterns  of  activity, 

4.  Modeling  of  Neuron  • 


4.1  General  Model.  A  variety  of  models  for  the  cortical  oscillations  found  in  die  visual  system 
have  been  proposed.  These  include  a  general  model  and  models  developed  by  H.  J.  Eckhom  and 
J .  L.  Johnson.  The  general  model  of  an  artificial  neural  net  (ANN)  is  an  artificial  neuron  (AN)  that 
receives  inputs  from  a  number  of  other  ANs  or  from  an  external  stimulus.  A  weighted  sum  of  these 
inputs  constitutes  the  argument  to  a  function.  The  resulting  value  of  the  function  is  the  output  of  die 
AN.  The  output  mimics  the  firing  of  a  biological  neuron.  This  ou^ut  gets  distributed  along 
weighted  connections  to  other  ANs.  Therefore,  an  artificial  neuron  anatomically  models  a  biological 
neuron. 

Looking  briefly  at  a  biological  neuron,  we  see  that  it  is  composed  of  dendrites,  a  cell  body,  an 
axon,  and  synaptic  buttons  (see  Figure  4)  (V ermuri  1992).  This  is  a  very  elementary  view  of  a  real 
neuron.  The  branching  stmcture  are  dendrites.  Dendrites  are  where  the  neuron  picks  up  signals  from 
other  neurons.  The  cell  body,  or  soma,  is  where  the  axon  hillock  acts  as  a  threshold  function.  The 
axon  hillock  determines  if  a  neuron  fires  (generates  an  action  potential)  based  on  a  comparison  of 
the  membrane  potential  and  the  threshold  value.  The  Icmg  transmission  line-like  structure  would  be 
the  axon,  and  the  action  potential  propagates  down  the  myelinated  axon  to  the  presynaptic  terminal. 
The  brushlike  structures  at  the  aid  of  the  axon  are  synaptic  buttons.  Synaptic  buttons  have  vesicles 
which  release  neurotransmitter  (NT)  from  the  presynaptic  terminal 
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Li  a  mathematical  abstraction  of  a  neuron,  XI— X„  represents  tiie  inputs  received  by  the  j*  neuron, 
Wji  represents  the  synaptic  strengths,  and  Yj  is  the  ou^ut  of  the  j*  neuron.  A  neuron  receives  signals 
from  its  neighbors  via  the  synapses  and  performs  a  weighted  algebraic  summation  on  the  inputs.  It 
computes  a  thresholding  function  and  produces  an  output  based  on  this  sum. 

A  diagram  of  a  neural  net  shows  the  points  where  neur(nis  come  close  together.  These  points 
form  the  synapses.  It  is  at  this  point  of  contact  that  the  synaptic  transmission  takes  place.  The 
neurons  influence  each  other  electrochemically.  When  a  signal  arrives  at  a  synapse,  it  elicits  the 
release  of  an  NT.  The  NT  changes  the  potential  of  the  neuron.  When  that  potential  exceeds  a  certain 
threshold,  an  action  potential  is  elicited  in  the  receiving  cell. 

A  biological  net  would  include  multiple  neurons  with  the  dendrites  making  multiple  connections 
between  neurons.  An  artificial  neural  net  would  thai  include  multiple  copies  of  the  modeled  neuron 
with  interconnecting  weights  and  the  output  that  is  determined  by  a  processing  element. 

This  is  the  basis  for  modeling  of  the  neuron  and  neural  net.  However,  each  component  of  the 
model  requires  detail.  Li  Eckhom’s  model,  the  processing  elements  are  developed.  He  discusses 
the  type  of  inputs  required,  the  subdividing  of  the  cell  body  into  compartments  that  perform  certain 
functions,  and  the  parameters  that  generate  an  output. 

Of  interest  are  two  models  that  recreate  the  neuronal  response  seal  in  the  studies  by  Gray  and 
Singer.  The  first  model  is  by  Eckhom  and  closely  recreates  the  responses  of  the  biological  system. 
The  second  model  is  by  Johnson.  Johnson’s  model  takes  Eckhom’s  model  and  attempts  to  modify 
it  to  use  the  patterns  of  the  neuronal  response  to  segment  images  in  a  visual  field. 

4.2  Eckhorn’s  Model.  Eckhom  discovered  that  stimulus-induced  oscillatory  activities 
(35—80  Hz)  in  cells  of  the  visual  cortex  synchronize  the  response  of  the  cell  if  a  common  stimulus 
drives  the  cells.  He  proposed  that  synchronization  is  a  general  principle  for  coding  sensory  systems 
and  diat  there  are  at  least  two  types  of  synchronizations.  These  synchronizations  are  stimulus-forced 
(event-locked)  synchronizations  and  stimulus-induced  (oscillatory)  synchronizations. 
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Eckhom  recorded  large  oscillation  amplitudes  of  LFPs  and  MUAs  fliat  were  induced  by  gngtairifd 
binocular  stimuli.  Stimulus-induced  oscillations  of  LFP  and  MUA  appeared  as  oscillation  spindles 
of  80-250  ms  long,  separated  by  intervals  of  stochastic  activity,  and  their  response  latencies  were 
longer  than  stimulus-locked,  visually  evoked  cortical  potentials  (VECP).  Stimulus-forced 
synchronizations  (VECPs)  are  drivoi  by  a  stimulus  that  is  rapidly  passed  across  the  field  of  view  and 
are  usually  not  oscillatory.  The  stimulus-specific  response  is  a  series  of  oscillating  voltages  at 
approximately  53  and  60  Hz  (see  Rgure  5a)  (Eckhom  et  al.  1989).  Stimulus-related  oscillations  of 
neural  activities  were  discovered  in  the  primary  visual  cortex  of  the  cat  through  studies  by  Gray  and 
Singer  (1987)  as  mentioned  earlier  and  by  Eckhom  et  al.  (1988).  Further  studies  by  Gray  and  Viana 
Di  Prisco  (1993)  indicated  that  stimulus-dependent  oscillations  of  30-60  Hz  have  been  confirmed 
in  single-electrode  recordings  in  A17  of  alert  cats.  These  findings  with  studies  by  Grossberg  (1983) 
and  Reitboeck  (1983)  support  die  hypothesis  that  synchronization  may  be  a  mechanism  for  carrying 
out  the  linking  of  local  visual  features  into  impressions  of  objects. 

Stimulus-specific  synchronization  was  observed  in  different  cortex  areas  if  the  neurons  coded 
common  visual  features.  As  mentioned  earlier  with  the  work  done  by  Gray  and  Singer,  signal 
oscillations  are  generated  in  the  cortex,  and  these  oscillations  can  be  synchronized  by  stimulation 
of  neurons  in  their  receptive  fields.  On  the  other  hand,  stimulus-induced  synchronizations  are 
recorded  as  oscillatory  mass  activities  and  are  assumed  to  be  produced  by  an  internal  mechanism. 
It  is  thought  that  this  internal  mechanism  is  the  process  achieved  from  stimulus-driven  local 
oscillators  that  are  mutually  coimected.  Simulations  have  been  done  to  support  this  assumption. 
These  stimulus-specific  oscillations  are  the  responses  modeled  by  Eckhom. 

The  stimulus-induced  synchronous  oscillatory  potential  found  to  take  place  in  the  visual  cortex 
were  observed  using  computer  simulations  of  neural  network  models.  Eckhom’s  model  has  two 
types  of  synapses:  (1)  the  feeding  synapses,  which  are  connected  directly  to  the  stimulus  that  drives 
the  neuron  and  (2)  the  linking  synapses  which  receive  auxiliary  signals  that  modulate  the  input  from 
the  feeding  synapses  (see  Figure  5b)  (Eckhom  et  al.  1989).  The  linking  inputs  are  considered  the 
synchronizing  signals. 
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Figure  5.  Stimulus-Specific  Synchronizations  in  Cat  Visual  Cortex  (a)  and  Eckhorn’s  Model 
of  Neuron  and  Neural  Net  (b). 
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The  model  neuron  has  djmamic  synapses  represented  by  leaky  integrators.  During  a  synaptic 
input  pulse,  the  integrator  is  charged  and  its  output  amplitude  rises  steeply.  This  is  followed  by  an 
exponential  decay  determined  by  what  is  called  the  leak  time  constant.  The  spike  encoder  is 
represented  by  a  leaky  integrator.  The  spike  encoder  includes  a  differential  amplitude  discriminator 
and  a  spike  former.  The  amplitude  discriminator  triggers  the  spike  former  when  the  input  exceeds 
a  variable  threshold.  The  ou^ut  of  the  neuron  inunediately  charges  the  leaky  integrator  anH  raises 
the  value  of  the  threshold.  This  new  threshold  value  exceeds  the  input,  preventing  an  output 
immediately  following  a  previous  ou^ut.  The  elevated  threshold  produces  the  refractory  period  in 
the  spike  generator.  Another  output  is  not  generated  until  the  threshold  value  decays  below  the 
membrane  potential. 

Eckhom  believes  that  the  concept  of  a  modulatory  synapse  is  supported  by  modulation  seen  in 
real  neurons.  These  modulations  may  be  achieved  by  changing  dendritic  membrane  potentials 
through  voltage-  dependent  channels  that  affect  synaptic  efficacy.  From  the  simnlatinns.  feeding 
signals  with  modulation  cause  the  model  to  respond  initially  with  irregular  receptive  discharges,  but 
subsequently,  the  neurons  mutually  synchronize  their  activity  through  the  linking  connections.  This 
supports  the  phase  locking  of  stimulus-induced  synchronizations  in  the  simulation. 


4^.1  Computational  Aspects.  Eckhom’s  neural  network  consist  of  two  layers  of  neurons.  The 
bottom  layer  (layer  1)  consists  of  several  neurons,  and  each  layer  1  neuron  receives  inputs  from  the 
stimulus  as  feeding  inputs  and  receives  input  from  the  four  closest  neighboring  neurons  as  linking 
inputs.  The  top  layer  (layer  2)  consists  of  several  neurons,  and  each  layer  2  neuron  receives  feeding 
inputs  from  the  four  closest  neurons  in  layer  1  and  receives  four  linking  inputs  from  its  four  closest 
neighboring  neuron  in  layer  2.  Furthermore,  each  layer  2  neuron  sends  feeding  inputs  to  the  four 
closest  layer  1  neurons.  Eckhom  models  his  individual  neuron  as  a  dendrite  and  soma,  with  the 
output  being  a  pulse  and  the  input  being  the  feeding  and  linking  inputs.  The  linking  inputs  and 
feeding  input  are  modeled  as  leaky  integrators.  Each  neuron  is  modeled  to  have  multiple  feeding 
and  linking  inputs.  The  linking  inputs  are  added  to  a  constant  value  of  one,  then  multiplied  with  the 
feeding  input  to  form  the  input  to  the  soma.  The  soma  is  modeled  as  a  spike  encoder.  The  spike 
encoder  is  where  the  output  from  the  dendrite  is  compared  with  a  preset  threshold  value,  fr  the  value 
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is  greater  than  the  tiireshold,  the  amplitude  discriminator  triggers  the  spike  former  to  output  a  pulse. 

This  output  is  fed  back  into  the  system  to  change  the  value  of  the  threshold  to  prevent  the  generating 
of  additional  pulses. 

The  equations  that  follow  describe  the  model  parameters  (see  Figure  6).  The  ouq)ut  of  the 
dendrite  is  the  membrane  voltage  U„^(t)  for  the  K*^  neuron: 

U^k(t)«[Fk(t)].[l+4(t)]. 

The  feeding  input: 

Fk(t)  -  i.  1  «,n2  [W^  ki  Yi(t)  +  Sk(t)  +  Nk(t)]  *  I  (v,T,t). 

The  linking  input: 

Lk(t)  -  i.  1  towS  [W*  ^  Yi(t)  +  Nk(t)]  *  I  (v,i:,t). 

The  threshold: 

0k(t)  -  ©0  +  Yk(t)  *  I  (v,T,t). 

The  output  of  the  spike  encoder  of  the  K*  neuron: 

Yk(t)-lifU„j,(t)k©,(t) 

Yk(t)  -  0  elsewhere. 

The  impulse  response  of  the  leaky  integraton 

I  (v,T,t)  -  V  •  exp(-t/T)  if  t  ^  0 
I  (v,T,t)  -  0  elsewhere, 
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The  membrane  voltage  of  the  A*th  neuron  is  given  by 

i/m>W  =  n-(0-[i  +  M0) 

(A.l) 

where  FiL.(/)  is  the  contribution  via  the  feeding-inputs  and  1,^ 
contribution  via  the  linking-inputs. 

Fu(t)  is  calculated  from 

(/)  is  the 

Fu{l)  =  E  [u'LyM  +  Sdi)  +  NUn]  *  r\l) 

tsi 

(A.2) 

impulse  response  of  leaky-integrators  at  the  feeding  inputs. 

(A.3) 

Tlie  contribution  via  linking-inputs  (index  "1")  is 

Ldi)=zHyM+f^dt)\*i{v\T\n 

»=i 

(A.5) 

The  output  of  the  spike-encoder  of  the  A  th  neuron  is  given  by: 

VfM-/  ’  U.nAt)  >  0,(1) 

'  \  0  else 

(A.6) 

The  threshold's  time  course  is  derived  by 

0,:(l)  =  0„+Y,{l.)*l(V\T\t) 

(A. 7) 

with  a  threshold-offset  /?(>• 

Figure  6.  Mathematical  Description  of  the  Model  Neuron  by  Eckhom. 


where 

N  is  the  number  of  neurons; 

u  is  the  synaptic  weight  of  the  feeding  input  from  the  i”*  to  the  k*  neuron; 

W*  u  is  the  synaptic  weight  of  the  linking  input  from  the  i*  to  the  k*  neuron; 

Yj(t)  is  the  analog  stimulus  input  to  the  k*  neuron  in  layer  1; 

Nk(t)  is  the  analog  “noise”  input  to  the  k*  neuron,  statistically  independent  in  each  neuron;  and 
©0  is  the  threshold  offset. 
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4^^  Simulations^  Eckhom  performed  several  tests  to  show  that  synchronization  can  be 
produced  with  his  model  (see  Figure  7).  The  first  experiment  is  with  two  stimuli  each  individually 
applied  to  two  patches  of  layer  1  neurons.  The  stimulus  applied  to  the  second  patch  was  half  the 
intensity  of  the  first  patch.  The  pulses  generated  in  layer  1  and  2  are  initially  at  different  frequencies 
for  each  patch.  When  feedback  (input  fix)m  the  linking  neurons)  is  not  included,  these  pulses  do  not 
lineup  (synchronize),  but  with  feedback  included,  we  observe  that  over  time  the  pulses  do  align, 
implying  synchronization.  Again,  the  experiment  is  done  by  applying  stimulus  to  one  patch  prior 
to  the  other,  pulses  are  again  generated  at  different  frequencies  for  the  patches.  Without  feedback, 
they  do  not  show  synchronization,  but  when  feedback  is  applied,  they  do  synchronize.  This  is  seen 
when  cross  correlation  is  monitored.  When  there  is  no  feedback,  there  is  no  cross  correlation 
recorded  for  the  two  patches.  However,  cross  correlation  is  recorded  when  feedback  is  presait  The 
experiments  are  repeated  with  stimulus  intensities  die  same;  the  stimulus  is  applied  to  both  patches, 
and  the  resulting  pulses  synchronize.  Then  die  stimulus  is  removed  from  one  patch.  With  pulses 
still  generated  from  each  patch,  the  resulting  pulses  synchronize  over  time.  The  pulses  generated 
from  the  patch  where  the  stimulus  was  removed  is  not  as  strong  as  the  pulses  generated  firom  the 
patch  where  the  stimulus  remains.  The  reason  why  the  pulse  from  the  patch  where  the  stimulus  is 
removed  still  appears  may  be  due  to  signals  that  are  sent  from  the  linking  inputs  corresponding  to 
the  four  neighboring  neurons. 

4J23  Behavioral  Aspects.  Limitations  ofEckhom’s  model  include  the  arbitrary  setting  of  the 
linking  weights  and  the  globally  setting  of  constants  to  experimentally  determined  values.  The 
constants  are  set  to  allow  the  model  simulations  to  closely  resemble  the  cats’  visual  cortex 
experimental  data.  These  constants  are  not  calculated  firom  biological  values. 

Eckhom  believes  his  model  supports  synchronization  as  a  way  to  achieve  perceptual  feature 
linking.  In  order  to  bring  neuronal  mechanisms  of  feature  linking  into  correspondence  with 
perceptual  functions,  Eckhom  introduces  the  concept  of  linking  field  of  a  local  neural  assembly.  The 
linking  field  of  the  local  neuronal  assembly  is  that  area  in  the  visual  space  where  appropriate  local 
stimulus  features  can  initiate  synchronizations  in  the  activities  of  that  assembly.  If  this  concept  can 
be  supported,  then  we  can  interpret  these  synchronized  patterns  of  neural  response  as  corresponding 
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Figure  7.  Neural  Network  Simulations. 
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to  visual  images.  It  has  been  seen  that  sensory  systems  that  received  well-timed  signals  might  be 
dominated  by  stimulus-forced  synchronization.  Other  systems  may  employ  a  mechanism  of 
synchronized  activity  for  oscillations  and  coding  in  the  system. 

4.3  Johnson’s  Model.  The  last  model  to  be  discussed  is  Johnson’s  model  (see  Figure  8a) 
(Johnson,  Ranagath,  and  Caulfield  1994;  Johnson  1994).  Johnson’s  model  does  not  correlate  with 
the  biological  system  as  strongly  as  Eckhom’s  model.  Johnson’s  model  is  a  modified  version  of 
Eckhom’s  model.  It  only  models  a  one-layer  network  but  with  multiple  recqitive  fields.  Johnson’s 
model  includes  a  dendritic-tree  section,  a  linking  section,  and  a  pulse-generator  section.  The 
dendritic  tree  is  similar  to  Eckhom’s  dendnte.  The  dendntic  tree  includes  the  feeding  inputs  from 
other  neurons,  as  well  as  the  linking  inputs  from  other  neurons.  The  difference  in  this  section  is  that 
the  linking  inputs  are  multiplied  by  a  variable  beta  before  the  constant  is  added.  Then  this  value  is 
multiplied  with  the  feeding  input  in  the  linking  section.  Moreover,  in  the  linking  section,  linking 
inputs  from  other  receptive  fields  are  multiplied  with  the  inputs  from  the  current  receptive  fields. 
This  value  is  then  summed  with  inputs  from  other  dendrites.  The  pulse-generator  section  includes 
a  threshold  discriminator  which  triggers  the  pulse  former  to  output  a  pulse  when  the  membrane 
potential  exceeds  the  threshold  value.  The  output  is  also  feedback  to  change  the  value  of  the 
threshold  as  in  Eckhom’s  model. 

43,1  Computational  Aspects,  It  is  helpful  to  look  at  an  individual  neuron  from  Johnson’s 
model  to  describe  what  functions  beta  and  tau-c  perform  (see  Figure  8b).  These  variables  are 
experimental  and  have  not  been  shown  to  model  biological  parameters.  Beta  is  called  the  weak- 
linking  variable.  Beta  sets  the  strength  of  the  linking  inputs’  effect  on  the  feeding  inputs.  Tau-c  is 
the  time  constant  that  defines  the  pulse-capture  zone.  This  zone  indicates  a  period  of  time  where  a 
linking  input  can  influence  the  frequency  of  the  ouq>ut  pulses.  Although  these  constants  are  not 
biologically  based,  they  do  play  an  important  role  in  the  model. 

hi  this  model,  when  there  is  no  feedback  from  the  neighboring  neurons  (no  linking  input),  beta 
(the  weak  linking  variable)  is  set  to  zero.  When  there  is  feedback  from  the  neighboring  neurons,  beta 
is  greater  than  zero.  To  understand  this  concept,  the  output  of  a  single  neuron  with  beta  equal  to  zero 
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Figure  8.  Johnson’s  Model  of  the  Neuron  and  the  Neural  Net  (a)  and  Neuron  Function  (b). 


23 


Figure  8.  Johnson’s  Model  of  the  Neuron  and  the  Neural  Net  (a)  and  Neuron  Function  (h)  (continued). 


is  examined.  With  beta  equal  to  zero,  the  membrane  potential  is  equal  to  the  feeding  input  only. 
When  the  membrane  potential  exceeds  the  threshold  value,  a  pulse  is  generated.  If  the  threshold 
value  is  increased  to  a  value  greater  than  the  membrane  potential,  then  the  threshold  value  decays 
with  time.  When  the  membrane  potential  exceeds  the  threshold  value  again,  then  another  pulse  is 
generated,  and  the  cycle  repeats.  This  time,  where  die  threshold  is  decaying  prevents  another  pulse 
from  generating  immediately  after  a  pulse,  thus  setting  the  refractory  period  or  the  pulse  period.  This 
is  the  same  way  Eckhom’s  model  operates.  Now  let  us  consider  when  beta  is  greater  than  zero.  A 
pulse  has  just  occurred,  the  threshold  value  is  increased  and  starts  to  decay,  ff  a  linking  input  is 
added  to  the  membrane  potential,  it  causes  the  value  of  the  membrane  potential  to  increase.  By  the 
same  token,  if  the  membrane  potential  is  greater  than  the  threshold  value,  a  pulse  is  generated.  But 
this  pulse  has  occurred  sooner  than  expected.  This  implies  that  the  linking  input  (or  the  feedback 
from  the  neighboring  neurons)  has  changed  when  the  pulse  occurs.  The  linking  input  synchronizes 
the  pulses  to  some  desired  pattern.  This  effect  of  the  linking  input  is  also  found  in  Eckhom’s  model; 
however,  the  linking  input  is  multiplied  by  a  constant  of  one,  not  beta.  Thus,  in  Johnson’s  model 
the  linking  inputs  for  the  current  neuron  can  be  weighted  differently  from  linking  inputs  of  neurons 
in  other  receptive  fields.  These  weights  are  set  arbitrarily  by  Johnson.  Since  these  weights  are  not 
from  biological  responses,  they  do  not  represent  synaptic  strengths  generated  by  a  specific  stimulus. 
Therefore,  this  model  does  not  use  these  weights  to  represent  data. 

Taking  this  concept  further,  Johnson  looks  at  four  individual  modeled  neurons,  each  neuron 
outputs  a  pulse  at  its  own  frequency  (see  Figure  9).  When  a  neuron  receives  a  linking  input  from 
another  neuron  within  the  tau-c  pulse-capture  zone,  die  neuron  generates  an  output  pulse  sooner  than 
it  normally  would  (i.e.,  the  ouqiut  pulse  fiiequency  is  increased).  Successive  interaction  through  the 
linking  inputs  readjust  the  pulses  from  each  neuron,  thus  synchronizing  them.  If  the  pulses  from  the 
four  neurons  are  added  together,  a  unique  pulse  train  is  formed.  The  pulse  train  is  then  considered 
the  response  for  a  particular  stimulus  or  image  in  the  field  of  view.  Each  image  generates  its  own 
pulse  train.  This  idea  could  be  justified  because  a  stimulus  does  produce  oscillations  in  the  neurons 
of  the  primary  visual  cortex.  Johnson’s  theory  also  assumes  that  this  pulse  train  repeats  over  some 
interval;  each  image  produces  a  periodic  pulse  train. 
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Input  Intensity  Scene 
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Figure  9.  Time  Series. 


43,2  Simulations.  Johnson  simulated  his  model  in  order  to  observe  if  any  meaningful 
could  be  extracted.  For  example,  Johnson  simulates  a  T  shape  with  varying  intensity  blocks  from 
white  to  black  (see  Figure  9).  The  image  produces  its  own  periodic  pulse  train.  The  fourth  and  fifth 
interval  and  presumably  the  following  intervals  would  be  identical.  The  simulation  shows  that  for 
the  first  four  intervals,  the  pulse  train  varies.  The  theoiy  is  that  these  pulse  trains  represent  changes 
as  the  linking  inputs  between  neurons  modulate  the  output  of  the  neurons;  this  is  then  the  time  where 
the  synchronization  is  occurring. 

The  pulse  train  is  generated  in  the  simulation  for  the  images  starting  at  the  highest  intensity 
section  and  propagating  to  the  section  with  the  lowest  intensity  (see  Figure  10a).  Thus,  if  you  look 
at  an  image  with  four  different  intensity  patterns,  each  pattern  produces  its  own  periodic  pulse  train 
or,  as  the  simulation  shows,  its  own  propagating  wave.  These  values  are  arbitrarily  set;  hence,  there 
are  no  data  to  show  what  the  interval  of  time  to  synchronize  might  be. 

4.33  Behavioral  Aspects.  Johnson  proposes  that  the  periodic  pulses  that  are  generated  by  a 
particular  stimulus  can  be  used  to  identify  segments  of  an  image  (see  Figure  10b).  F,ach  small 
section  of  an  image  would  produce  its  individual  pulse  train.  As  the  field  of  view  is  expanded,  it 
would  produce  its  individual  pulse  train  until  the  entire  scene  is  covered.  The  smaller  pulse  trains 
would  actually  become  subgroups  within  the  pulse  trains  produced  by  the  larger  section  of  the  scene. 
Therefore,  you  would  have  a  supergroup  pulse  train  that  would  represent  the  entire  scene  and  would 
consist  of  subgroups  of  pulse  trains  at  each  level  of  the  expanded  view. 

5.  Conclusion  and  Issues 

This  report  has  presented  three  models  that  attempt  to  describe  the  oscillations  produced  in  the 
primary  visual  cortex.  Even  diough  fliese  models  do  not  model  the  biological  system,  they  do  allow 
exploration  into  the  response  of  the  biological  system.  Modeling  the  oscillations  found  in  the  visual 
cortex  has  provided  a  possible  tool  for  segmenting  images. 
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(b) 

Figure  10.  Segmentation  of  Patterns  (a)  and  Subgroups  and  Supergroups  of  Pulses  (b). 
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Synchronous  oscillations  are  found  in  the  visual  cortex.  However,  more  investigation  would  help 
determine  how  these  oscillations  correspond  to  various  stimulus.  Stimulus-induced  oscillations 
seem  to  indicate  that  feedback  increases  the  pulse  frequency  and  causes  pulses  to  occur  periodically. 
However,  interaction  between  the  feeding  and  linking  inputs  must  be  examined  more  closely  to 
understand  this  function.  Moreover,  the  weak  linking  variable  and  the  pulse  capture  zone  concept 
used  to  create  periodic  pulse  trains  needs  to  be  investigated  to  determine  its  link  with  the  biological 
system.  Eckhom’s  and  Johnson’s  models  can  be  viewed  as  first-stage  systems.  These  models  use 
the  oscillatory  property  that  can  be  achieved  in  modeling  of  neural  networks  to  reproduce  the 
response  of  neurons  found  in  the  visual  cortex. 

The  work  done  by  Eckhom  and  Johnson  raises  several  interesting  questions.  Some  of  these 
questions  include: 

(1)  Are  responses  to  stimuli  the  same  for  different  levels  of  alertness? 

(2)  How  useful  are  the  models  as  they  move  away  from  biology  (the  beta  and  tau-c  experimental 
values)? 

(3)  How  could  the  individual  periodic  pulse  trains  that  correlate  to  segmented  objects  be 
filtered? 

Obviously  these  questions  cover  a  wide  range— from  the  way  the  biological  system  truly 
functions  to  the  way  the  simulations  could  be  validated.  Addressing  these  issues  should  provide  a 
better  understanding  into  cortical  oscillations  and  the  way  they  can  be  used  in  other  levels  of 
recognition. 

Further  modeling  can  be  done  to  incorporate  more  of  the  functionality  of  the  real  biological 
system.  Thus,  if  Johnson’s  model  can  be  validated  biologically,  then  inhibitory  neurons  should  be 
modeled  and  added  to  the  overall  network  model.  These  inhibitory  neurons  can  possibly  define 
where  the  boundaries  of  the  propagating  waves  are  and,  thus,  define  the  boundaries  for  segmentation 
of  objects. 
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